Verbal IQ of a Four-Year Old Achieved by an AI System

One view of common-sense reasoning ability is that it is the ability to perform those tasks with verbal inputs and outputs that have traditionally been difficult for computer systems, but are easy for fairly young children. We administered the verbal part of the Wechsler Preschool and Primary Scale of Intelligence (WPPSI-III, Third Edition) to the ConceptNet 4 system. The IQ test’s questions (e.g., “Why do we shake hands?” or “What do apples and bananas have in common”) were translated into ConceptNet 4 inputs using a combination of the simple natural language processing tools that come with ConceptNet together with short Python programs that we wrote. The question-answering primarily used the part of the ConceptNet system that represents the knowledge as a matrix based on spectral methods (AnalogySpace). We found that the system has a Verbal IQ that is average for a four-year-old child, but below average for 5, 6, and 7 yearolds. Large variations from subtest to subtest indicate potential areas of improvement. In particular, results were strongest for the Vocabulary and Similarities subtests, intermediate for the Information subtest, and lowest for the Comprehension and Word Reasoning subtests. Comprehension is the subtest most strongly associated with common sense. Children’s verbal IQ tests offer a new, objective, third-party metric for the evaluation and comparison of common-sense AI systems.